A Malleable Interface for Sonic Exploration

Input devices for controlling music software can benefit from exploiting the use of perceptual-motor skill in interaction. The project described here is a new musical controller, designed with the aim of enabling intuitive and nuanced interaction through direct physical manipulation of malleable material. The controller is made from conductive foam. This foam changes electrical resistance when deformed; the controller works by measuring resistance at multiple points in a single piece of foam in order to track its shape. These measurements are complex and interdependent so an echo state network, a form of recurrent neural network, is employed to translate the sensor readings into usable control data. A cube shaped controller was built and evaluated in the context of the haptic exploration of sound synthesis parameter spaces. Eight participants experimented with the controller and were interviewed about their experiences. The controller achieves its aim of enabling intuitive interaction, but in terms of nuanced interaction, accuracy and repeatability were issues for some participants. It’s not clear from the short evaluation study whether these issues would improve with practice, a longitudinal study that gives musicians time to practice and find the creative limitations of the controller would help to evaluate this fully. The evaluation highlighted interesting issues concerning the high level nature of malleable control and different approaches to sonic exploration.